{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# PyTorch 基础 : 张量\n",
    "在第一章中我们已经通过官方的入门教程对PyTorch有了一定的了解，这一章会详细介绍PyTorch 里面的基础知识。\n",
    "全部掌握了这些基础知识，在后面的应用中才能更加快速进阶，如果你已经对PyTorch有一定的了解，可以跳过此章"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'0.4.1'"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 首先要引入相关的包\n",
    "import torch\n",
    "import numpy as np\n",
    "#打印一下版本\n",
    "torch.__version__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 张量(Tensor)\n",
    "张量的英文是Tensor，它是PyTorch里面基础的运算单位,与Numpy的ndarray相同都表示的是一个多维的矩阵。\n",
    "与ndarray的最大区别就是，PyTorch的Tensor可以在 GPU 上运行，而 numpy 的 ndarray 只能在 CPU 上运行，在GPU上运行大大加快了运算速度。\n",
    "\n",
    "下面我们生成一个简单的张量"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0.0053, 0.6125, 0.8095],\n",
       "        [0.8511, 0.8716, 0.4374]])"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = torch.rand(2, 3)\n",
    "x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "以上生成了一个，2行3列的的矩阵，我们看一下他的大小："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([2, 3])\n",
      "torch.Size([2, 3])\n"
     ]
    }
   ],
   "source": [
    "# 可以使用与numpy相同的shape属性查看\n",
    "print(x.shape)\n",
    "# 也可以使用size()函数，返回的结果都是相同的\n",
    "print(x.size())\n",
    "# print(x.item()) # 只有含一个元素的tensor才能用tensor.item()取出其中的具体数值"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "张量（Tensor）是一个定义在一些向量空间和一些对偶空间的笛卡儿积上的多重线性映射，其坐标是|n|维空间内，有|n|个分量的一种量， 其中每个分量都是坐标的函数， 而在坐标变换时，这些分量也依照某些规则作线性变换。r称为该张量的秩或阶（与矩阵的秩和阶均无关系）。 (来自百度百科)\n",
    "\n",
    "下面我们来生成一些多维的张量："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([2, 3, 4, 5])\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "tensor([[[[0.8695, 0.4509, 0.1183, 0.2049, 0.7025],\n",
       "          [0.9709, 0.5072, 0.6466, 0.5610, 0.7098],\n",
       "          [0.7790, 0.2172, 0.1020, 0.3249, 0.8566],\n",
       "          [0.5279, 0.5230, 0.5646, 0.8199, 0.0553]],\n",
       "\n",
       "         [[0.8734, 0.7861, 0.0783, 0.6437, 0.2349],\n",
       "          [0.4570, 0.4023, 0.0006, 0.5439, 0.8253],\n",
       "          [0.7711, 0.3352, 0.7990, 0.4580, 0.4476],\n",
       "          [0.5652, 0.9170, 0.3912, 0.4686, 0.7118]],\n",
       "\n",
       "         [[0.8011, 0.6186, 0.0040, 0.1023, 0.2234],\n",
       "          [0.9753, 0.2945, 0.1710, 0.2645, 0.4000],\n",
       "          [0.5632, 0.7224, 0.1974, 0.8013, 0.4679],\n",
       "          [0.7644, 0.0647, 0.9560, 0.4820, 0.8850]]],\n",
       "\n",
       "\n",
       "        [[[0.9833, 0.1566, 0.4308, 0.8394, 0.2651],\n",
       "          [0.4526, 0.0077, 0.6511, 0.0751, 0.6748],\n",
       "          [0.4986, 0.6003, 0.6874, 0.9061, 0.2693],\n",
       "          [0.1290, 0.5192, 0.3672, 0.9233, 0.7115]],\n",
       "\n",
       "         [[0.7241, 0.4223, 0.3510, 0.9763, 0.8954],\n",
       "          [0.1375, 0.8768, 0.1664, 0.9429, 0.5177],\n",
       "          [0.8294, 0.7315, 0.1755, 0.1223, 0.8680],\n",
       "          [0.0054, 0.3417, 0.3899, 0.1493, 0.9682]],\n",
       "\n",
       "         [[0.2414, 0.0526, 0.8918, 0.6344, 0.3352],\n",
       "          [0.6530, 0.1174, 0.0470, 0.9904, 0.2784],\n",
       "          [0.0806, 0.4079, 0.5559, 0.3337, 0.4763],\n",
       "          [0.5433, 0.7638, 0.8019, 0.4376, 0.6044]]]])"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y=torch.rand(2,3,4,5)\n",
    "print(y.size())\n",
    "y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在同构的意义下，第零阶张量 （r = 0） 为标量 （Scalar），第一阶张量 （r = 1） 为向量 （Vector）， 第二阶张量 （r = 2） 则成为矩阵 （Matrix），第三阶以上的统称为多维张量。\n",
    "\n",
    "其中要特别注意的就是标量，我们先生成一个标量：\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(3.1433)\n",
      "torch.Size([])\n"
     ]
    }
   ],
   "source": [
    "#我们直接使用现有数字生成\n",
    "scalar =torch.tensor(3.1433223)\n",
    "print(scalar)\n",
    "#打印标量的大小\n",
    "print(scalar.size())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "对于标量，我们可以直接使用 .item() 从中取出其对应的python对象的数值"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3.143322229385376"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "scalar.item()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "特别的：如果张量中只有一个元素的tensor也可以调用`tensor.item`方法"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([3.1433])\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "torch.Size([1])"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tensor = torch.tensor([3.1433223]) \n",
    "print(tensor)\n",
    "tensor.size()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3.143322229385376"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tensor.item()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 基本类型\n",
    "Tensor的基本数据类型有五种：\n",
    "- 32位浮点型：torch.FloatTensor。 (默认)\n",
    "- 64位整型：torch.LongTensor。\n",
    "- 32位整型：torch.IntTensor。\n",
    "- 16位整型：torch.ShortTensor。\n",
    "- 64位浮点型：torch.DoubleTensor。\n",
    "\n",
    "除以上数字类型外，还有\n",
    "byte和chart型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([3])"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "long=tensor.long()\n",
    "long"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([3.1426], dtype=torch.float16)"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "half=tensor.half()\n",
    "half"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([3], dtype=torch.int32)"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "int_t=tensor.int()\n",
    "int_t"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([3.1433])"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "flo = tensor.float()\n",
    "flo"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([3], dtype=torch.int16)"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "short = tensor.short()\n",
    "short"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([3], dtype=torch.int8)"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ch = tensor.char()\n",
    "ch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([3], dtype=torch.uint8)"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bt = tensor.byte()\n",
    "bt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Numpy转换\n",
    "使用numpy方法将Tensor转为ndarray"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[-1.1113302  -0.3842171 ]\n",
      " [ 0.49684364  1.9792417 ]\n",
      " [-2.41278    -0.44195625]]\n"
     ]
    }
   ],
   "source": [
    "a = torch.randn((3, 2))\n",
    "# tensor转化为numpy\n",
    "numpy_a = a.numpy()\n",
    "print(numpy_a)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "numpy转化为Tensor\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[-1.1113, -0.3842],\n",
       "        [ 0.4968,  1.9792],\n",
       "        [-2.4128, -0.4420]])"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch_a = torch.from_numpy(numpy_a)\n",
    "torch_a"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***Tensor和numpy对象共享内存，所以他们之间的转换很快，而且几乎不会消耗什么资源。但这也意味着，如果其中一个变了，另外一个也会随之改变。***"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 设备间转换\n",
    "一般情况下可以使用.cuda方法将tensor移动到gpu，这步操作需要cuda设备支持"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'torch.FloatTensor'"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cpu_a=torch.rand(4, 3)\n",
    "cpu_a.type()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'torch.cuda.FloatTensor'"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gpu_a=cpu_a.cuda()\n",
    "gpu_a.type()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "使用.cpu方法将tensor移动到cpu"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'torch.FloatTensor'"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cpu_b=gpu_a.cpu()\n",
    "cpu_b.type()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如果我们有多GPU的情况，可以使用to方法来确定使用那个设备，这里只做个简单的实例："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cuda\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'torch.cuda.FloatTensor'"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#使用torch.cuda.is_available()来确定是否有cuda设备\n",
    "device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n",
    "print(device)\n",
    "#将tensor传送到设备\n",
    "gpu_b=cpu_b.to(device)\n",
    "gpu_b.type()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 初始化\n",
    "Pytorch中有许多默认的初始化方法可以使用"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0.9737, 0.9267, 0.6361],\n",
       "        [0.8592, 0.2742, 0.8456],\n",
       "        [0.9543, 0.7462, 0.3553],\n",
       "        [0.8272, 0.4115, 0.1774],\n",
       "        [0.0595, 0.2068, 0.0069]])"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 使用[0,1]均匀分布随机初始化二维数组\n",
    "rnd = torch.rand(5, 3)\n",
    "rnd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[1., 1.],\n",
       "        [1., 1.]])"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "##初始化，使用1填充\n",
    "one = torch.ones(2, 2)\n",
    "one"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0., 0.],\n",
       "        [0., 0.]])"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "##初始化，使用0填充\n",
    "zero=torch.zeros(2,2)\n",
    "zero"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[1., 0.],\n",
       "        [0., 1.]])"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#初始化一个单位矩阵，即对角线为1 其他为0\n",
    "eye=torch.eye(2,2)\n",
    "eye"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 常用方法\n",
    "PyTorch中对张量的操作api 和 NumPy 非常相似，如果熟悉 NumPy 中的操作，那么 他们二者 基本是一致的："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[-0.7995,  0.7521,  0.1759],\n",
      "        [-2.0889, -0.9356, -1.4167],\n",
      "        [ 0.1856, -0.3047, -0.5738]])\n"
     ]
    }
   ],
   "source": [
    "x = torch.randn(3, 3)\n",
    "print(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([ 0.7521, -0.9356,  0.1856]) tensor([1, 1, 0])\n",
      "tensor([0.1856, 0.7521, 0.1759]) tensor([2, 0, 0])\n"
     ]
    }
   ],
   "source": [
    "# 沿着行取最大值\n",
    "max_value, max_idx = torch.max(x, dim=1)\n",
    "print(max_value, max_idx)\n",
    "# 沿列取最大值\n",
    "max_value, max_idx = torch.max(x, dim=0)\n",
    "print(max_value, max_idx)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([ 0.1286, -4.4412, -0.6929])\n",
      "tensor([-2.7028, -0.4882, -1.8146])\n"
     ]
    }
   ],
   "source": [
    "# 每行 x 求和\n",
    "sum_x = torch.sum(x, dim=1)\n",
    "print(sum_x)\n",
    "# 每列 x 求和\n",
    "sum_x = torch.sum(x, dim=0)\n",
    "print(sum_x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[-0.3821, -2.6932, -1.3884],\n",
      "        [ 0.7468, -0.7697, -0.0883],\n",
      "        [ 0.7688, -1.3485,  0.7517]])\n"
     ]
    }
   ],
   "source": [
    "y=torch.randn(3, 3)\n",
    "z = x + y\n",
    "print(z)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "正如官方60分钟教程中所说，以_为结尾的，均会改变调用值"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[-0.3821, -2.6932, -1.3884],\n",
      "        [ 0.7468, -0.7697, -0.0883],\n",
      "        [ 0.7688, -1.3485,  0.7517]])\n"
     ]
    }
   ],
   "source": [
    "# add 完成后x的值改变了\n",
    "x.add_(y)\n",
    "print(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "张量的基本操作都介绍的的差不多了，下一章介绍PyTorch的自动求导机制"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
